Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(kafka): Fix internal docker connection #2894

Open
wants to merge 59 commits into
base: main
Choose a base branch
from

Conversation

mdelapenya
Copy link
Member

@mdelapenya mdelapenya commented Nov 21, 2024

What does this PR do?

Follow-up of #2490 in order to be able to push commits and make progress.

Why is it important?

Give importance to @catinapoke's work in #2490, keeping the commits (and authorship), and doing the toil tasks for them. This way we can push to this feature branch (the original PR is using the main branch, so we cannot update it)

Related issues

@mdelapenya mdelapenya requested a review from a team as a code owner November 21, 2024 15:44
@mdelapenya mdelapenya self-assigned this Nov 21, 2024
@mdelapenya mdelapenya requested a review from stevenh November 21, 2024 15:45
@mdelapenya mdelapenya added the bug An issue with the library label Nov 21, 2024
Copy link

netlify bot commented Nov 21, 2024

Deploy Preview for testcontainers-go ready!

Name Link
🔨 Latest commit 67c3480
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-go/deploys/67447ef10348ba00083ff40a
😎 Deploy Preview https://deploy-preview-2894--testcontainers-go.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@mdelapenya mdelapenya changed the title fix(kafka): Fix kafka internal docker connection fix(kafka): Fix internal docker connection Nov 21, 2024
Copy link
Collaborator

@stevenh stevenh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could use the same approach as see in the this #2883?

docs/modules/kafka.md Outdated Show resolved Hide resolved
modules/kafka/kafka.go Outdated Show resolved Hide resolved
modules/kafka/kafka.go Outdated Show resolved Hide resolved
modules/kafka/kafka.go Outdated Show resolved Hide resolved
modules/kafka/kafka.go Outdated Show resolved Hide resolved
modules/kafka/kafka_test.go Outdated Show resolved Hide resolved
Comment on lines 16 to 20
func defaultOptions() options {
return options{
Listeners: make([]KafkaListener, 0),
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: I suspect this isn't needed.

modules/kafka/options.go Show resolved Hide resolved
modules/kafka/options.go Outdated Show resolved Hide resolved
@@ -525,7 +525,8 @@ func TestRedpandaListener_NoNetwork(t *testing.T) {
func TestRedpandaBootstrapConfig(t *testing.T) {
ctx := context.Background()

container, err := redpanda.RunContainer(ctx,
container, err := redpanda.Run(ctx,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: this should be a separate PR as its a different module.

@mdelapenya
Copy link
Member Author

@stevenh I addressed the majority of your comments: all except those related to the TestKafka_networkConnectivity test, which is still failing for me locally.

// Consumer represents a Sarama consumer group consumer
type TestConsumer struct {
t *testing.T
ready chan bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: its idiomatic to use a struct{} for this pattern

Suggested change
ready chan bool
ready chan struct{}


return TestConsumer{
t: t,
ready: make(chan bool),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ready: make(chan bool),
ready: make(chan struct{}),

@@ -38,6 +38,12 @@ type KafkaContainer struct {
ClusterID string
}

type Listener struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Add doc comment.

brokers, err := kafkaContainer.Brokers(context.TODO())
require.NoError(t, err)

// err = createTopics(brokers, []string{topic_in, topic_out})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: remove commented out code.

if ctx.Err() != nil {
return
}
consumer.ready = make(chan bool)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug: this looks racy to me as its overwriting the read which is already set in the call to NewTestConsumer

t.Log("Sarama consumer up and running!...")

cancel()
wg.Wait()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: is the waitgroup and ready doing the same thing?

if errors.Is(err, sarama.ErrClosedConsumerGroup) {
return
}
require.NoError(t, err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug: you can't fail a test in a goroutine.


func validateListeners(listeners ...Listener) error {
// Trim
for i := 0; i < len(listeners); i++ {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: doesn't the caller needed the update listeners to work correctly?

Comment on lines +105 to +106
listeners[i].Host = strings.Trim(listeners[i].Host, " ")
listeners[i].Port = strings.Trim(listeners[i].Port, " ")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: is this just space char or should this use strings.TrimSpace?

listeners[i].Name = strings.ToUpper(strings.Trim(listeners[i].Name, " "))
listeners[i].Host = strings.Trim(listeners[i].Host, " ")
listeners[i].Port = strings.Trim(listeners[i].Port, " ")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: as there's no constructor hence these fields could be blank do we need to test for that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug An issue with the library
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants